Home | | Latest | About | Random
# 11 Some basic matrix properties.
Let us record down some properties of these matrix operations (sum, product, transposes, and scaling).
## Some properties regarding matrix sum.
First let us record some results regarding matrix sum and scaling.
> **Properties of matrix sums.**
> Suppose $A,B,C$ are matrices of the same size (say all $n\times k$), and let $r,s$ be scalars. Then
> (1) $A+B=B+A$
> (2) $(A+B)+C=A+(B+C)$
> (3) $A+O_{n\times k}=A$
> (4) $r(A+B)=rA + rB$
> (5) $(r+s)A=rA + sA$
> (6) $r(sA)=(rs)A$
Each of these properties are very believable, and they can all be "named". For example, provided that the sizes of the matrices pose no issue, then (1) matrix addition is commutative; (2) matrix addition is associative; (3) the zero matrix is an additive identity; (4) scaling distributes over matrix addition; (5) matrix scaling distributes over scalar sums; (6) matrix scaling is associative.
To prove each of these statements, again one can check that on each side of the equal sign, they have the same entries. For instance,
$\blacktriangleright$ Proof of (1). Note the $(i,j)$-th entry of $A+B$ is $$
\begin{align*}
(A+B)_{ij} & = (A)_{ij}+(B)_{ij} & \text{definition of matrix addition} \\
& =(B)_{ij} + (A)_{ij} & \text{scalar addition is commutative} \\
& =(B + A)_{ij} & \text{definition of matrix addition}
\end{align*}$$ which is the $(i,j)$-th entry of $B+A$. Hence $A+B = B+A$. $\blacksquare$
$\blacktriangleright$ Proof of (4). Note that $(i,j)$-th entry of $r(A+B)$ is $$
\begin{align*}
[r(A+B)]_{ij} & = r(A+B)_{ij} & \text{definition of matrix scaling} \\
& =r((A)_{ij}+(B)_{ij}) & \text{definition of matrix addition} \\
& = r(A)_{ij} + r(B)_{ij} & \text{scalar distributes over scalar sum} \\
& = (rA)_{ij} + (rB)_{ij} & \text{definition of matrix scaling} \\
& = (rA+rB)_{ij} & \text{definition of matrix sum} \\
\end{align*}
$$ which is the $(i,j)$-th entry of $rA+rB$. $\blacksquare$
One can similarly prove the rest of above using these basic definitions of matrix sum and scaling, and properties of scalars (numbers). You should try mimicking writing some of these down.
## Some properties regarding matrix product.
Now let us record some results regarding matrix product.
> **Properties of matrix products.**
> Let $A$ be an $n\times k$ matrix, and let $B,C$ be matrices with sizes so that each of the following matrix product is defined, and let $r$ be a scalar. Then we have
> (1) $A(BC)=(AB)C$
> (2) $A(B+C)=AB+AC$
> (3) $(B+C)A=BA+CA$
> (4) $r(AB)=(rA)B=A(rB)$
> (5) $I_{n}A=A=AI_{k}$
We proved (one of the equalities of) (5) in [[1 teaching/smc-spring-2024-math-13/linear-algebra-notes/10-some-special-matrices|notes 10]] when we talked about identity matrices. I will leave (2) to (4) for you to prove as exercise (try using definition of matrix product, and analyze entry by entry as we have so far).
What may be not so believable is (1) the associativity of matrix product, perhaps because we know matrix product is not commutative in general. There is a deeper reason why matrix product is associative. But we will prove it entry by entry as we have before, using the definition of matrix product.
$\blacktriangleright$ Proof of (1) associativity of matrix product.
Let $A$ be an $n\times k$ matrix, $B$ be $k\times p$, and $C$ be $p\times q$. We shall demonstrate that $A(BC)=(AB)C$ by showing the left hand side has the same $(i,j)$-th entry as the right hand side. Note $$
\begin{align*}
[A(BC)]_{ij} & = \sum_{t=1}^{k}(A)_{it}(BC)_{tj} \\
& =\sum_{t=1}^{k} (A)_{it} \left(\sum_{s=1}^{p}(B)_{ts}(C)_{sj}\right) \\
& =\sum_{t=1}^{k} \sum_{s=1}^{p} (A)_{it} (B)_{ts} (C)_{sj} \\
& =\sum_{s=1}^{p} \sum_{t=1}^{k} (A)_{it} (B)_{ts} (C)_{sj} \\
& =\sum_{s=1}^{p}\left( \sum_{t=1}^{k} (A)_{it} (B)_{ts} \right) (C)_{sj} \\
& =\sum_{s=1}^{p} (AB)_{is} (C)_{sj} \\
& =[(AB)C]_{ij}
\end{align*}
$$by repeatedly using the definition of matrix product, and properties of finite summation. $\blacksquare$
**Remark.** Note that when we move the summation sign around above, or interchange them, we have to be careful with the "bounds" and the dummy variables (much like for double-integrals). Since these bounds are "nice", we can move them around like that.
**Remark 2.** Perhaps you have been wondering why matrix multiplication is defined in such a "peculiar" way, so the fact that this definition of matrix product gives rise to associativity of matrix product shows "the definition is not so arbitrary" after all.
**Remark 3.** Some warnings. Matrix multiplication does not behave like numbers exactly. Try to find examples of the following:
- In general $AB\neq BA$ for matrices $A,B$.
- Cancellation rule does not always hold, that is, if $AB=AC$, it does not necessarily imply $B=C$.
- If $AB=O$, some zero matrix, it does not imply $A$ or $B$ is a zero matrix.
## Some properties regarding matrix transpose.
Now let us record some results regarding matrix transposes. An important one being the "sock-and-shoe" result.
> **Properties of matrix transpose.**
> Let $A$ and $B$ be matrices with sizes such that the sums and products described below are defined, and let $r$ be a scalar. Then,
> (1) $(A^{T})^{T}$
> (2) $(A+B)^{T}=A^{T}+B^{T}$
> (3) $rA^{T}=(rA)^{T}$
> (4) $(AB)^{T}= B^{T}A^{T}$
The last one is what I call a "sock-and-shoe" result. I will leave (1) to (3) as exercises to you. For (4), let us prove it, using the same idea as before.
$\blacktriangleright$ Proof of (4). Let $A$ be some $n\times k$ matrix, and $B$ be some $k\times p$ matrix. We will show that the $(i,j)$-th entry of $(AB)^{T}$ to be the same as the $(i,j)$-th entry of $B^{T}A^{T}$. Note, $$
\begin{align*}
((AB)^{T})_{ij} & =(AB)_{ji} & \text{definition of transpose}\\
& = \sum_{t=1}^{k}(A)_{jt}(B)_{ti} & \text{definition of matrix product}\\
& = \sum_{t=1}^{k} (A^{T})_{tj} (B^{T})_{it} & \text{definition of transpose}\\
& = \sum_{t=1}^{k} (B^{T})_{it} (A^{T})_{tj} & \text{scalar multiplication is commutative}\\
& =(B^{T}A^{T})_{ij} & \text{definition of matrix product}
\end{align*}
$$And we have $(AB)^{T}=B^{T}A^{T}$ as desired. $\blacksquare$
**Remark.** So here, the transpose of a product is the product of the transposes in reverse order. Can you explain why this is a "sock-and-shoe" situation?